Removal of specularities using color and polarization

Specular reflections and interreflections produce strong highlights in brightness images. These highlights can cause vision algorithms, such as, segmentation, shape from shading, binocular stereo, and motion detection to produce erroneous results. We present an algorithm for separating the specular and d i f i s e components of reflection from images. The method iises color and polarization, simultaneously, to obtain strong constraints on the reflection components at each image point. Polarization is used to locally determine the color oj the specular component, constraining the difSuse color at a pixel to a one dimensional linear subspace. This subspact> is used to find neighboring pixels whose color is consistent with the pixel. Diffuse color information from consistent neighbors is used to determine the diffuse color of the pixel. In contrast to previous separation algorithms, the proposed method can handle highlights that have a varying diffuse component as well as highlights that include regions with diffkrent reflectance and material properties. We present several txperimental results obtained by applying the algorithm to complex scenes with textured objects and strong